📙 Vocabulary Reference

Last updated on August 7, 2024 by Sander Schulhoff

Please refer to this page for a list of terms and concepts that we will use throughout this guide.

Large Language Models (LLMs), Pretrained Language Models (PLMs), Language Models (LMs), and foundation models

These terms all refer more or less to the same thing: large AIs (neural networks), which have usually been trained on a huge amount of text.

"Model" vs. "AI" vs. "LLM"

These terms are used somewhat interchangeably throughout this course, but they do not always mean the same thing. LLMs are a type of AI, but not all AIs are LLMs. When we mention models in this course, we are referring to AI models. As such, in this course, you can consider the terms "model" and "AI" to be interchangeable.

Machine Learning (ML)

ML is a field of study that focuses on algorithms that can learn from data. ML is a subfield of AI.

Prompt

A prompt is an input to a LLM to guide its output. In the context of LLMs, a prompt is usually a piece of text, but given different modalities (meaning that the model can work with input other than text data, like images or audio), prompts are not limited to just text.

Prompt Chain

A prompt chain is a sequence of two or more prompt templates used in succession. For example, one prompt template might be used, and then the output of that template goes as the input of the next template.

Prompt Engineering

Prompt Engineering is the iterative process of developing a prompt by modifying or changing the prompting technique that you are using.

Prompt Engineering Technique

A prompt engineering technique is a strategy for iterating on a prompt to improve it. In production or academic settings, this is often automated, but in consumer settings, it is often manually.

Prompt Template

Prompts are often constructed using prompt templates. A prompt template contains 1 or more variables that are filled in with specific values to create a prompt. For example, a prompt template for sentiment analysis might be "Classify this tweet as positive or negative: [TWEET]". The variable [TWEET] is filled in with the content of a tweet to create a prompt. Note that templates are not limited to text.

Prompting

Prompting is the process of providing a prompt to an LLM to guide its output. For example, the action of inputting a chunk of text or uploading an image constitutes prompting.

Prompting Technique

A prompting technique is a blueprint that describes how to structure a prompt, prompts, or dynamic sequencing of multiple prompts. A prompting technique may incorporate conditional logic, parallelism, or other architectural considerations spanning multiple prompts.

Zero-Shot Prompts

Zero-Shot Prompts contain no examples or exemplars in their input prompt. Zero-Shot Chain-of-Thought is an example of a Zero-Shot prompting technique.

Few-Shot Learning

Few-shot learning is a machine learning paradigm that aims to increase the accuracy of a model by training on a small number of examples. This is not to be confused with Few-Shot Prompting, which more specifically refers to the prompting technique.

Few-Shot Prompting

Few-shot Prompting is a technique for LLMs to generate more desired output after first being prompted with a few examples. Design decisions behind few-shot prompting are as follows:

• Exemplar selection -- The method of selecting the few examples to prompt the model.
• Exemplar ordering -- The method of ordering the examples to prompt the model.
• Exemplar number -- The amount of examples to prompt the model with. Generally more examples are better, but there are diminishing returns (around 20).
• Exemplar label quality -- The quality of examples provided. The necessity of high quality examples is unclear, as some work suggests that providing models with incorrect labels of exemplars may not hurt performance.
• Input distribution -- How much of each label to provide to the model.
• Input-label pairing format -- The Formatting of exemplars. One common format is "Q: input, A: label", but the optimal format may vary across tasks.

Chain-of-Thought (CoT) Prompting

Chain-of-Thought Prompting is a few-shot prompting technique in which the user feeds the LLM a prompt that demands a thought process before giving the final answer. Usually, an example of the thought process is given before asking for an answer (hence the few-shot nature of the prompt). A common example is:

Q: Jack has two baskets, each containing three balls. How many balls does Jack have in total?
A: One basket contains 3 balls, so two baskets contain 3 * 2 = 6 balls.
Q: [the question for inference] 
A:

Context

The context of a prompt is additional background information given to LLMs to help generate better outputs. For example, context can also refer to the tokens processed by the LLM in a forward pass.

In-Context Learning (ICL)

In-Context Learning is the ability of an LLM to learn new skills from its provided prompt. These skills can be learned from exemplars or instructions. Few-shot prompting is also called In-Context Learning.

Analogical Prompting

Analogical Prompting enhances the problem-solving capabilities of LLMs by initially prompting them to generate relevant concepts and questions, and afterwards the LLM addresses the original prompt. For each question generated, the LLM is tasked with both describing the problem and explaining the solution. This approach has demonstrated notable improvements in mathematical reasoning and code generation tasks, outperforming Zero-Shot and Few-Shot CoT methods.

Contrastive Chain-of-Thought

Contrastive Chain-of-Thought or Contrastive CoT involves adding incorrect explanations to the CoT prompt alongside the correct reasoning in order to show the LLM how not to reason. This method has shown significant improvement over CoT in areas like arithmetic reasoning and factual question answering.

Decomposition Techniques

Decomposition based prompting techniques share the fundamental premise that breaking down a complex problem into smaller, more manageable subproblems can improve the performance of LLMs. Some decomposition techniques include DECOMP and Least-to-Most Prompting.

Decomposed Prompting (DECOMP)

Decomposed Prompting is similar to Least-to-Most Prompting, but uses few-shot Chain-of-Thought (CoT) to guide the LLM in breaking down complex problems into sub-problems. This modular framework accommodates diverse decomposition structures, including recursion and hierarchical decomposition, besides the linear left-to-right decomposition of Least-to-Most Prompting. When evaluated on various programming and question-answering tasks, DECOMP surpasses the Least-to-Most method in performance.

Labels

The concept of labels is best understood with an example.

Say we want to classify some Tweets as mean or not mean. If we have a list of Tweets and their corresponding label (mean or not mean), we can train a model to classify whether tweets are mean or not. Labels are generally just possibilities for the classification task.

Label Space

The space of all possible labels for a given task.

Least-to-Most Prompting

Least-to-Most Prompting is a technique where the LLM first decomposes a problem into smaller sub-problems, then solves these sequentially to arrive at the final answer. This method has shown significant improvements in tasks involving symbolic manipulation, compositional generalization, and mathematical reasoning. However, it’s important to note that the prompts for decomposition do not universally apply across different problems. The effectiveness hinges on correctly breaking down the problem, which may not consistently occur with a fixed prompt.

Masked Language Models (MLMs)

MLMs are a type of NLP model, which have a special token, usually [MASK], which is replaced with a word from the vocabulary. The model then predicts the word that was masked. For example, if the sentence is "The dog is [MASK] the cat", the model will predict "chasing" with high probability.

N-Shot Prompting (Zero-Shot, One-Shot, Few-Shot, etc)

This family of terms refers to the number of examples used to prompt a model. For example, Zero-Shot Prompting is when a model is prompted with no examples, and Few-shot Prompting is when a model is prompted with a few examples. Prompting a model with a few examples can be a good way to improve performance on a task without needing to fine-tune the model.

Output Formatting

Output formatting is the general idea of guiding an LLM to produce its output in a specific format. One example is outputting in CSV format, where the model produces a comma-separated list of values.

Plan-and-Solve Prompting

Plan-and-Solve enhances Zero-Shot CoT with prompts like "Let’s first understand the problem and devise a plan to solve it. Then, let’s carry out the plan and solve the problem step by step". This method generates more robust reasoning processes than standard Zero-Shot CoT and even rivals the performance of various Few-Shot CoT methods.

Prompt Mining

Prompt Mining is a technique used to algorithmically identify the best prompt template for a given corpus of text

API

Application Programming Interface. Enables different systems to interact with each other programmatically. Two types of APIs are REST APIs (web APIs) and native-library APIs. See What is a REST API?

Reinforcement Learning from Human Feedback (RLHF)

Reinforcement Learning from Human Feedback is a method for fine tuning LLMs according to human preference data.

Role

A role, also referred to as persona, is a role or character a user instructs the model to identify as.

Role Prompting

Role Prompting is a technique in which the user assigns the model a role or persona to improve performance. For example, a user might ask the model to solve a math problem as a math teacher, or to write a poem as a poet.

Step-Back Prompting

Step-Back prompting is a modification of CoT where the LLM is first asked a generic, high-level question about relevant concepts or facts before delving into reasoning. This has been shown to improve performance in MMLU tasks.

Style Prompting

Style prompting is a technique in which the user assigns the model a style to improve performance. For example, a user might ask the model to write a poem in the style of Shakespeare. This can also be done using role prompting.

Thought Generation

Thought generation is a family of techniques that aim to guide the model to generate thoughts or ideas. This can be done using a variety of prompting techniques, such as Thread-of-Thought.

Thread-of-Thought (ThoT)

Thread-of-Thought uses the phrase "Walk me through this context in manageable parts step by step, summarizing and analyzing as we go." in CoT prompting to enhance question answering and retrieval capabilities.

Verbalizer

In the classification setting, verbalizers are mappings from labels to words in a language model's vocabulary. For example, consider performing sentiment classification with the following prompt:

Here, the verbalizer is the mapping from the conceptual labels of positive and negative to the tokens pos and neg.

Sentiment Analysis

Sentiment analysis is the task of classifying text into positive, negative, or other sentiments.

Self-Consistency

Self-Consistency is generating multiple chains of thought and taking the majority answer.

Program-aided Language Model

Program-aided Language Model(PAL) is a method that uses code as intermediate reasoning.

Priming

Priming refers to the method of using the first prompt to a chatbot to set the structure and style of a conversation.

Debiasing

Debiasing, or Prompt Debiasing involves applying specific methods to ensure that LLM responses are not skewed toward certain biases.

Jailbreaking

Jailbreaking is the act of getting a GenAI model to perform or produce unintended outputs through specific prompts. We also wrote a whole article about what Jailbreaking is and how it is different from Prompt Injection: Prompt Injection VS Jailbreaking: What is the difference?

Footnotes

1. Branch, H. J., Cefalu, J. R., McHugh, J., Hujer, L., Bahl, A., del Castillo Iglesias, D., Heichman, R., & Darwishi, R. (2022). Evaluating the Susceptibility of Pre-Trained Language Models via Handcrafted Adversarial Examples. ↩

2. Schulhoff, S., Ilie, M., Balepur, N., Kahadze, K., Liu, A., Si, C., Li, Y., Gupta, A., Han, H., Schulhoff, S., & others. (2024). The Prompt Report: A Systematic Survey of Prompting Techniques. arXiv Preprint arXiv:2406.06608. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷

3. Brown, T. B. (2020). Language models are few-shot learners. arXiv Preprint arXiv:2005.14165. ↩ ↩² ↩³

4. Wu, T., Terry, M., & Cai, C. J. (2022). Ai chains: Transparent and controllable human-ai interaction by chaining large language model prompts. Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, 1–22. ↩

5. Shin, T., Razeghi, Y., Logan IV, R. L., Wallace, E., & Singh, S. (2020). Autoprompt: Eliciting knowledge from language models with automatically generated prompts. arXiv Preprint arXiv:2010.15980. ↩

6. Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022). Large Language Models are Zero-Shot Reasoners. ↩

7. Fei-Fei, L., Fergus, R., & Perona, P. (2006). One-shot learning of object categories. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(4), 594–611. ↩

8. Wang, Y., Yao, Q., Kwok, J. T., & Ni, L. M. (2020). Generalizing from a few examples: A survey on few-shot learning. ACM Computing Surveys (Csur), 53(3), 1–34. ↩

9. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022). Chain of Thought Prompting Elicits Reasoning in Large Language Models. ↩

10. Yew Ken Chia. (2023). Contrastive Chain-of-Thought Prompting. In arXiv preprint arXiv:1907.11692. ↩ ↩²

11. Fu, Y., Peng, H., Sabharwal, A., Clark, P., & Khot, T. (2022). Complexity-based prompting for multi-step reasoning. The Eleventh International Conference on Learning Representations. ↩

12. Yasunaga, M., Chen, X., Li, Y., Pasupat, P., Leskovec, J., Liang, P., Chi, E. H., & Zhou, D. (2023). Large language models as analogical reasoners. arXiv Preprint arXiv:2310.01714. ↩

13. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., & others. (2019). Language models are unsupervised multitask learners. OpenAI Blog, 1(8), 9. ↩

14. Tushar Khot. (2023). Decomposed Prompting: A Modular Approach for Solving Complex Tasks. ↩

15. Li, C., Wang, J., Zhang, Y., Zhu, K., Hou, W., Lian, J., Luo, F., Yang, Q., & Xie, X. (2023). Large language models understand and can be enhanced by emotional stimuli. arXiv Preprint arXiv:2307.11760. ↩

16. Zhou, D., Schärli, N., Hou, L., Wei, J., Scales, N., Wang, X., Schuurmans, D., Cui, C., Bousquet, O., Le, Q., & Chi, E. (2022). Least-to-Most Prompting Enables Complex Reasoning in Large Language Models. ↩

17. Lei Wang. (2023). Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models. ↩

18. Schmidt, D. C., Spencer-Smith, J., Fu, Q., & White, J. (2023). Cataloging prompt patterns to enhance the discipline of prompt engineering. URL: Https://Www. Dre. Vanderbilt. Edu/Undefined̃ Schmidt/PDF/ADA_Europe_Position_Paper. Pdf [Accessed 2023-09-25]. ↩

19. Wang, Z., Mao, S., Wu, W., Ge, T., Wei, F., & Ji, H. (2024). Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration. https://arxiv.org/abs/2307.05300 ↩

20. Zheng, M., Pei, J., & Jurgens, D. (2023). Is “A Helpful Assistant” the Best Role for Large Language Models? A Systematic Evaluation of Social Roles in System Prompts. https://arxiv.org/abs/2311.10054 ↩

21. Zheng, H. S., Mishra, S., Chen, X., Cheng, H.-T., Chi, E. H., Le, Q. V., & Zhou, D. (2023). Take a step back: Evoking reasoning via abstraction in large language models. arXiv Preprint arXiv:2310.06117. ↩

22. Lu, A., Zhang, H., Zhang, Y., Wang, X., & Yang, D. (2023). Bounding the capabilities of large language models in open text generation with prompt constraints. arXiv Preprint arXiv:2302.09185. ↩

23. Liu, J., Liu, A., Lu, X., Welleck, S., West, P., Bras, R. L., Choi, Y., & Hajishirzi, H. (2021). Generated Knowledge Prompting for Commonsense Reasoning. ↩

24. Zhou, Y., Geng, X., Shen, T., Tao, C., Long, G., Lou, J.-G., & Shen, J. (2023). Thread of thought unraveling chaotic contexts. arXiv Preprint arXiv:2311.08734. ↩

25. Schick, T., & Schütze, H. (2020). Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference. ↩

26. Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., & Zhou, D. (2022). Self-Consistency Improves Chain of Thought Reasoning in Language Models. ↩

27. Liu, J., Shen, D., Zhang, Y., Dolan, B., Carin, L., & Chen, W. (2022). What Makes Good In-Context Examples for GPT-3? Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures. https://doi.org/10.18653/v1/2022.deelio-1.10 ↩

28. Gao, L., Madaan, A., Zhou, S., Alon, U., Liu, P., Yang, Y., Callan, J., & Neubig, G. (2023). Pal: Program-aided language models. International Conference on Machine Learning, 10764–10799. ↩